Mining the Web with Active Hidden Markov Models

Given the enormous amounts of information available only in unstructured or semi-structured textual documents, tools for information extraction (IE) have become enormously important. IE tools identify the relevant information in such documents and convert it into a structured format such as a database or an XML document. While first IE algorithms were hand-crafted sets of rules, researchers soon turned to learning extraction rules from hand-labeled documents. Unfortunately, rule-based approaches sometimes fail to provide the necessary robustness against the inherent variability of document structure, which has led to the recent interest in the use of hidden Markov models (HMMs) [1] for this purpose. Speech recognition and computational biochemistry are well-known applications of HMMs. Markov model algorithms that are used for part-ofspeech tagging, as well as known hiddenMarkovmodels for information extraction [1] require the training documents to be labeled completely, i.e., each token is manually given an appropriate semantic label. Clearly, this is an expensive process. We therefore concentrate on the task of learning hidden Markov models, from partially (sparsely) labeled texts, and develop appropriate EM-style algorithms. By using additional unlabeled documents as they are usually readily available in most applications, we can perform active learning of HMMs. The idea of active learning algorithms is to identify unlabeled observations that would be most useful when labeled by the user. Such algorithms are known for classification, clustering, and regression; we present the first algorithm for active learning of hidden Markov models.